The Effect of Dependency Representation Scheme on Syntactic Language Modelling

There has been considerable work on syntactic language models and they have advanced greatly over the last decade. Most of them have used a probabilistic contextfree grammar (PCFG) or a dependency grammar (DG). In particular, DG has attracted more and more interest in the past years since dependency parsing has achieved great success. While much work has evaluated the effects of different dependency representations in the context of parsing, there has been relatively little investigation into them on a syntactic language model. In this work, we conduct the first assessment of three dependency representations on a transition-based dependency parsing language model. We show that the choice of dependency representation has an impact on overall performance from the perspective of language modelling.